* Ignore _reqs, _plans, _docs working dirs

Local feature-planning artifacts (requirements docs, implementation
plans, scratch notes) live under these prefixes per the
feature-planning skill convention. They're per-developer working
notes, not shipped artifact, so they stay out of git.

* Suppress Slack webhooks during tests

notifier._send() calls load_dotenv() on every invocation, which
loads the developer's real SLACK_WEBHOOK_URL from .env and POSTs
live messages during test runs whenever pipeline code paths hit
an end-of-batch notification. Tests must never fire real external
side effects.

Add an autouse conftest fixture that strips SLACK_WEBHOOK_URL and
patches notifier._load_dotenv to None for the test session. Unit
test runtime drops from ~5.5s to ~3s as a side benefit.

* Add service-mode package scaffolding

Phase 1 of the service-mode work (see _plans/service-mode-plan.md).
Lay down the empty src/service/ package and its module stubs so
subsequent phases can fill them in without dependency churn.

New dependencies (fastapi, uvicorn[standard], httpx) are added to
the main runtime requirements. Pydantic, click, and rich are
already present and unchanged.

Adds a second project script entry, audio-refinery-service, that
will become the long-lived HTTP service entrypoint once Phase 7
lands. The existing audio-refinery CLI is unchanged; service mode
is additive.

* Scope GitHub Actions workflows to least privilege

CodeQL flags both workflows for missing top-level permissions
blocks. The default GITHUB_TOKEN grants more than these jobs need.

Set workflow-level default to contents:read for ci.yml and
release.yml. The github-release job in release.yml retains its
existing job-level contents:write override to publish releases.
Tests, lint, type-check, and build jobs only read source.

Closes CodeQL alerts #1 through #5.

* Upgrade vulnerable transitive dependencies

Refresh the lockfile to pull in security fixes for transitive deps
that uv was previously holding at older versions:

  idna     3.11    -> 3.15    (CVE-2024-3651 follow-up)
  Mako     1.3.10  -> 1.3.12  (path traversal fixes)
  Pillow   12.1.1  -> 12.2.0  (PSD OOB write + others)
  urllib3  2.6.3   -> 2.7.0   (redirect header forwarding + decompression bomb)

Targeted upgrades only — torch and transformers stay pinned at
their WhisperX-compatible versions (2.1.2 and >=4.30,<4.40). Those
Dependabot alerts close together when v0.3.0 drops WhisperX.

All 216 unit tests pass against the upgraded versions.

Closes Dependabot alerts #21, #22, #23, #24, #25, #26, #27, #28, #29.

* Document security work in Unreleased changelog

Capture the workflow-permissions scoping, transitive-dep upgrades,
and the test-suite Slack suppression under the [Unreleased] section.
Also call out the known security debt against torch and transformers
that stays open until v0.3.0 drops WhisperX.

* Implement service-mode URI I/O

Phase 2 of the service-mode work. Adds fetch_input() and upload()
helpers that handle both https:// (presigned, via httpx) and file://
(local-disk, with parent-dir creation on upload) schemes.

fetch_input streams HTTPS bodies to a destination path and returns
the destination; for file:// it verifies the source exists and
returns the original path without copying so the worker reads it
in-place from the bind-mounted location.

upload accepts a JSON-serializable dict and PUTs it (HTTPS) or
writes it to disk (file://). The same function serves both per-job
transcript uploads and the per-batch summary upload.

Custom UnsupportedScheme / FetchError / UploadError exceptions map
the underlying failures into a service-layer surface that the
worker exception path can react to cleanly.

21 new unit tests; full unit suite 237/237 passing.

* Implement combined transcript and batch summary schemas

Phase 3 of the service-mode work. Adds the two service-layer
Pydantic documents:

CombinedTranscript wraps the existing per-stage results
(DiarizationResult + TranscriptionResult + optional SentimentResult)
into one document per successful job, uploaded to the caller's
output_uri. Includes a model_versions block for quick lookup and
top-level schema_version for forward-compatibility when v0.3.0
alignment changes the shape.

BatchSummary captures per-job terminal outcomes for the whole
batch, written to the caller-supplied summary_uri after every job
settles. Per-job entries carry status="completed" (with
duration_seconds) or status="failed" (with stage, error,
retryable). Totals are derived from the job entries by the
factory so callers don't track them separately.

Both schemas start at schema_version 1.0.0 in this release.

build_combined() and build_summary() factories assemble the
documents from caller-supplied inputs; no coupling to the Job
dataclass that lands in Phase 5.

11 new unit tests; full unit suite 248/248 passing.

* Split service schemas and factories into separate modules

The Phase 3 work landed schemas plus their factory functions in one
file (src/service/transcript.py), mixing pure data with assembly
behavior. The rest of the project's models follow the convention
that Pydantic data lives in dedicated files (src/models/*.py) while
factories live in the modules that have other reasons to exist.

Apply the same convention within src/service/:

- src/service/schemas.py (renamed from transcript.py): pure
  Pydantic data only — CombinedTranscript, JobSummaryEntry,
  BatchTotals, BatchSummary, JobStatus and JobFailureStage type
  aliases, schema version constants. No imports of factory helpers,
  no side effects.

- src/service/jobs.py: schema factories build_combined() and
  build_summary() (plus the _audio_refinery_version helper that
  reads importlib.metadata). The worker and registries land here
  in Phase 5; the factories are co-located now so they're already
  next to their natural caller when the worker arrives.

The rename also fixes a content-vs-name mismatch: the old
"transcript.py" held both the transcript and the batch summary
schemas, so "schemas.py" describes what's in the file.

tests/service/test_transcript.py renamed to test_schemas.py;
imports updated to source schemas from schemas and factories from
jobs. CLAUDE.md, docs/DEVELOPMENT.md project-structure trees and
the implementation plan refreshed to reference the new layout.

All 248 unit tests pass against the new module layout.

* Refactor pipeline for service-mode pre-loaded model handles

Phase 4 of the service-mode work. Adds the model lifecycle pieces
the long-lived container needs so it can pay each stage's model
load cost once at startup rather than on every job.

src/service/lifecycle.py:
- PipelineHandles dataclass bundles pre-loaded pyannote, WhisperX,
  and (optionally) sentiment handles.
- ServiceConfig dataclass captures container-startup config
  resolved from env vars (device, models, compute_type, sentiment
  toggle, HF token).
- ServiceReadiness is the thread-safe state object the /health
  endpoint reads. Three states: loading, ready, failed (with the
  offending stage and detail).
- warm_up(config, readiness) orchestrates the in-order load of
  diarization -> transcription -> sentiment. On any failure it
  raises WarmupError carrying the stage name, and updates the
  readiness object so /health can attribute the failure.
- start_thermal_guard ports the CLI's _run_temp_guard pattern
  (src/cli.py:126) into the service: a daemon thread polls GPU
  temperature every 5s and invokes a caller-supplied on_trip
  callback when the limit is reached. Returns a threading.Event
  the caller flips to terminate the guard cleanly during shutdown.
  Returns None for cpu device or when limit <= 0.
- default_thermal_trip fires the existing notify_thermal_shutdown
  Slack notification and calls os._exit(1). Phase 5 will replace
  this with a worker-aware callback that also marks the in-flight
  job failed and writes the partial batch summary.

src/pipeline.py:
- run_pipeline() accepts a new optional model_handles parameter.
  When supplied the pipeline reuses the caller's handles via the
  existing _pipeline= / _whisperx_model= / _sentiment_pipeline=
  injection seams and skips the internal load entirely. When None
  (CLI default) behavior is exactly as before.

17 new lifecycle tests plus 2 new pipeline injection tests cover
warmup orchestration, readiness state, thread-safety, thermal
guard behavior (trips/skips/cpu/disabled), and the run_pipeline
handle-injection path. Full unit suite: 267/267.

* Make thermal-guard polling configurable via ServiceConfig

Add gpu_temp_limit_celsius and gpu_temp_poll_seconds fields to
ServiceConfig so the thermal guard's threshold and polling cadence
are first-class config rather than hard-coded at the call site.

The new start_thermal_guard_from_config(config, on_trip) helper
threads those two fields plus config.device into the underlying
start_thermal_guard call. Phase 7's lifespan handler calls this
once at startup; direct callers (tests, custom integrations) can
still use start_thermal_guard with explicit args.

Default poll cadence stays at 5.0 seconds to match the CLI's
existing _run_temp_guard behavior. Operators on shared hosts where
nvidia-smi polling is expensive can bump REFINERY_GPU_TEMP_POLL_SECONDS
to back off.

3 new tests cover the wrapper: field threading, disabled-config
returns None, configured poll interval actually reaches the daemon
thread (verified via mocked temperature query and timing).

Full unit suite: 270/270.

* Add structlog dep, worker config, and notify_job_failed

Phase 5a: foundation pieces the worker needs in subsequent commits.

- structlog>=24.0 added to runtime deps. Logger configuration lands
  in Phase 7's app.py lifespan handler; the worker will import and
  use structlog from Phase 5c.
- ServiceConfig gains three new fields tied to env vars the worker
  honors: intermediate_dir (REFINERY_INTERMEDIATE_DIR), max_queue_size
  (REFINERY_MAX_QUEUE_SIZE, default 100), and job_retention_seconds
  (REFINERY_JOB_RETENTION_SECONDS, default 3600).
- notifier.notify_job_failed(job_id, stage, input_uri, error) is the
  Slack hook the worker fires on per-job failure. Presigned URL query
  strings are stripped before the message is sent so the channel stays
  legible. Successful jobs continue to fire nothing — the
  failure-only policy is enforced at the call sites in Phase 5c.

* Add in-memory data primitives for service-mode jobs

Phase 5b: dataclasses, registries, queue, and ID-generation helpers
the worker will operate on in Phase 5c.

- Job dataclass tracks a single job's lifecycle: queued ->
  processing -> completed | failed. Carries timestamps,
  failure detail (stage, error, retryable), and duration on
  success.
- Batch dataclass tracks a whole batch: batch_id, summary_uri,
  job_ids, and a pending_count that the worker decrements as
  jobs settle.
- JobRegistry and BatchRegistry are thread-safe in-memory dicts.
  BatchRegistry.decrement_pending atomically decrements and
  returns the new value so the worker can detect the
  zero-transition without a separate read; never goes negative.
- JobQueue wraps queue.Queue with a configurable maxsize.
  Over-capacity put_nowait raises queue.Full, which the HTTP
  endpoint will translate into 429 Too Many Requests in Phase 7.
- Registries bundle ties the three together for the worker and
  endpoint constructors.
- make_job_id() returns "rfj_<16-hex>" via secrets.token_hex(8);
  make_batch_id() returns "btc_<16-hex>". 64 bits of entropy
  per id, collision-resistant for any realistic deployment.

22 new tests cover identifier shape, registry CRUD,
thread-safety under contention, decrement atomicity across
8 threads, queue FIFO ordering, and queue cap enforcement.
Full unit suite: 295/295.

* Implement service-mode worker, process_job, and finalize_batch

Phase 5c: the actual job-processing loop and batch finalization.

process_job(job, handles, config, registries) runs one job end to
end: marks the job as processing, downloads or symlinks the input
into a per-job temp dir (no copy for file:// inputs), invokes
run_pipeline with the supplied warm handles, inspects per-stage
outcomes for the content_id, deserializes the per-stage JSONs the
pipeline wrote, assembles a CombinedTranscript via build_combined,
and uploads to job.output_uri. On any failure (download error,
unreachable file, stage failure, upload error, uncaught exception)
the worker records the failure detail on the in-memory Job record,
fires notify_job_failed via the failure-only Slack hook, and never
writes anything to job.output_uri. Failures collapse into the
batch summary as the canonical failure surface.

When REFINERY_INTERMEDIATE_DIR is set on the config, the worker
copies the per-stage JSONs from the temp dir to <dir>/<job_id>/
after the upload succeeds. Best-effort: copy failures log a
structured warning and do not fail the job.

finalize_batch(batch_id, registries) reads every terminal Job from
the batch's job_ids, converts each into a JobSummaryEntry, builds
a BatchSummary via build_summary, and uploads to the
caller-supplied summary_uri. Summary upload failures log but do
not crash the worker — the orchestrator's CloudWatch idle-time
alarm catches "summary never appeared" as the backstop.

Worker class wraps the queue.get -> process_job -> mark_terminal ->
maybe-finalize loop in a daemon thread. One worker per container —
GPU is the bottleneck. Uncaught exceptions inside process_job are
absorbed by the top-level handler so the worker thread never dies;
the affected job is marked failed and the loop continues.

External dependencies (run_pipeline, fetch_input, upload,
notify_job_failed) are looked up at module scope so tests patch
via the standard patch("src.service.jobs.<name>") seam. structlog
emits structured events at every state transition; logger
configuration lands in Phase 7's lifespan handler.

11 new tests cover the worker behavior in isolation: happy path,
download / transcribe / upload / uncaught-exception failure paths,
optional intermediate persistence, finalize_batch upload success
and failure, Worker class end-to-end with queue, worker continues
after one job fails in a batch, worker stop terminates the thread.

Full unit suite: 306/306.

* Add RetentionSweeper for terminal job and batch records

Phase 5d: the periodic cleanup that keeps the in-memory registries
from growing unboundedly over the container's lifetime.

RetentionSweeper is a background daemon that ticks every 60s
(configurable). On each tick it evicts every terminal Job
(status completed or failed, completed_at/failed_at older than
config.job_retention_seconds) and every terminal Batch
(completed_at older than the same window). Pending jobs and
in-flight batches are never touched.

After eviction GET /jobs/{id} returns 404, matching the
integration contract's "absence is not a contract violation"
clause. Orchestrators that need to find old outcomes already have
the per-batch summary in object storage at that point — the
per-job in-memory record is just a polling-convenience surface.

Best-effort: a sweep_once exception logs a structured warning and
the thread keeps ticking. Never dies on its own.

Exposed sweep_once as a public method so tests and a future admin
endpoint can trigger an immediate sweep without waiting for the
next tick.

6 new tests cover terminal-job eviction, terminal-batch eviction,
empty-registry sanity, thread lifecycle, end-to-end sweep via the
daemon thread, and continue-after-error resilience.

Full unit suite: 312/312. Phase 5 complete.

* Implement bearer-token auth middleware

Phase 6: validates Authorization: Bearer <key> against an env-loaded
allowlist (REFINERY_API_KEYS). One tier of API keys — "is this caller
allowed to use Refinery at all" — decoupled from the storage-auth
story (presigned URLs handle bucket access on the caller's side).

- load_allowlist_from_env() reads REFINERY_API_KEYS, splits on comma,
  strips whitespace, drops empty entries. Raises AllowlistError on
  empty/unset so the container fails fast at startup.
- fingerprint(token) returns the first 8 hex chars of SHA-256(token).
  Stable across calls, one-way, suitable for audit-log correlation
  without leaking the bearer.
- make_bearer_dependency(allowlist) returns a FastAPI Depends-compatible
  callable that validates the bearer scheme + token. Returns the
  fingerprint on success so route handlers can include it in structured
  logs. Raises HTTPException(401) with {"error": "invalid_bearer"} on
  any failure (missing header, wrong scheme, empty token, not in
  allowlist) — the same response shape regardless of which check
  failed so callers can't distinguish failure modes via timing or
  body.

The allowlist set is frozen at dependency-construction time so later
mutations to the caller's set don't change auth behavior — a
deliberate guard against accidental privilege escalation.

20 new tests: fingerprint properties (stable, distinct, opaque,
8-hex chars), allowlist parsing edge cases (whitespace, empty
entries, unset, alternative env var), direct dependency invocation
(valid/invalid token, missing header, wrong scheme, empty token,
frozen-against-mutation), and four end-to-end TestClient round-trips
including the route-scoped behavior that lets /health stay
unauthenticated alongside protected /transcribe and /jobs routes.

Full unit suite: 332/332.

* Implement FastAPI app, lifespan, and endpoints

Phase 7: wires every prior service-mode commit into the actual HTTP
surface that satisfies the integration contract.

src/service/app.py:
- create_app(config, *, registries, readiness, api_keys, handles,
  enable_lifespan_warmup) — pure factory. Production callers
  (run()) pass only config; tests inject pre-built state. State
  attrs are set at construction so endpoints work whether or not
  the lifespan handler has run.
- TranscribeRequest / JobRequest Pydantic models with URI scheme
  validation via src.service.uri_io.validate_scheme.
- POST /transcribe (auth): per-config batch cap (default 25),
  queue-capacity precheck (429 on full), registers Job records
  first, Batch second, then enqueues so the worker never pulls
  a job whose record isn't in the registry.
- GET /jobs/{job_id} (auth): 404 with {"error":"job_not_found"}
  on miss; full status body otherwise.
- GET /health (no auth): 200 when ready, 503 with stage+detail
  while loading or failed.
- Lifespan handler: pre-supplied handles -> starts Worker +
  RetentionSweeper synchronously. Otherwise spawns warm_up() in
  a background daemon thread so /health stays reachable as
  503/loading while the ~10s model load runs.
- _StructLogContextMiddleware binds http_method + http_path into
  structlog contextvars so every log record processed during a
  request includes them — without the Authorization header.
- _configure_structlog wires JSON renderer (default) or console
  renderer (REFINERY_LOG_FORMAT=console) for local dev.
- run() is the `audio-refinery-service` entry point: configures
  structlog, validates the allowlist (fails fast on empty), builds
  the app, runs uvicorn binding to all interfaces on REFINERY_PORT.

src/service/lifecycle.py:
- ServiceConfig gains max_batch_size: int = 25 to match the new
  server-side batch cap in POST /transcribe.

21 new tests cover the full HTTP surface: /health states
(loading/ready/failed, unauthenticated), /transcribe happy paths
and validation (missing summary_uri, empty jobs, bad scheme on
each URI, batch-too-large), auth + queue-full 429, /jobs happy
and 404 + auth paths, and a lifespan smoke test confirming Worker
+ RetentionSweeper start when handles are supplied. Full unit
suite: 353/353.

* Extract HTTP transport schemas to src/service/api_schemas.py

Phase 7's app.py had Pydantic request/response models inline, mixing
pure data with framework wiring. That's the same pattern Phase 3's
schemas split moved away from.

Move the 5 transport schemas (JobRequest, TranscribeRequest,
TranscribeResponse, JobStatusResponse, HealthResponse) and the
_validate_uri helper into a new src/service/api_schemas.py.
app.py imports them and keeps only the framework wiring
(endpoints, lifespan, middleware, run() entrypoint).

Two schema files now exist with clearly different scopes:

  schemas.py     — content (what we serialize to disk: transcript +
                   batch summary). Evolution gated by what the
                   pipeline produces.
  api_schemas.py — transport (what flows over the wire). Evolution
                   gated by the integration contract in
                   _docs/refinery-integration.md.

Keeping the two layers separate means a transcript-schema bump
(e.g., v0.3.0 alignment splitting aligned words into a separate
array) doesn't churn the HTTP wire format, and vice versa.

Tests unchanged — they hit the HTTP endpoints, not the models
directly. Full unit suite: 353/353.

* Extract ServiceConfig and PipelineHandles to src/service/config.py

Apply the same data-vs-behavior split the rest of the package now
follows. ServiceConfig and PipelineHandles are pure data — a frozen
dataclass and a mutable bundle of pre-loaded model handles — that
previously lived in lifecycle.py alongside warm_up(),
ServiceReadiness, the thermal guard, and the default_thermal_trip
callback. Mixing pure data with the behavior that consumes it
violates the same convention we've now applied to schemas.py vs
jobs.py and api_schemas.py vs app.py.

New src/service/config.py holds:
  - ServiceConfig (16 frozen fields covering device, models, queue,
    retention, thermal guard, and worker behavior).
  - PipelineHandles (diarization, whisperx, optional sentiment).
  - No behavior, no imports of anything that runs side effects.

lifecycle.py now holds only behavior: warm_up(), ServiceReadiness
state machine, WarmupError, start_thermal_guard(*),
default_thermal_trip. Imports the two dataclasses from config.

Touched importers across the package and test suite:
  src/pipeline.py        — TYPE_CHECKING import path updated.
  src/service/app.py     — split import into config + lifecycle.
  src/service/jobs.py    — import from config.
  tests/service/         — three files updated.
  tests/test_pipeline.py — both injection-path tests.

CLAUDE.md and docs/DEVELOPMENT.md project-structure trees list the
new module.

Full unit suite: 353/353. mypy clean across all 9 service modules.

* Rewrite Dockerfile for service mode

The existing Dockerfile defaulted to `audio-refinery --help` and
used the CUDA devel base image. Service mode replaces both.

Dockerfile changes:
- Base swap: nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04
  -> nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04. Drops the
  full CUDA SDK we don't need at runtime; ~3GB image-size win.
- syntax=docker/dockerfile:1.7 header for BuildKit features.
- apt install gets --no-install-recommends so the image stays
  lean. ffmpeg + curl + git remain in place.
- All Python/whisperx/torch install steps carry over verbatim,
  with `--system` flags on uv pip install so packages land in
  the container's system Python (no venv).
- EXPOSE 8000 declares the service port.
- HEALTHCHECK polls /health every 10s after a 60s start-period.
  Start-period covers the ~10s warmup before /health flips from
  503 (status=loading) to 200 (status=ok); 60s leaves headroom
  for slower GPU loads and avoids flapping the container.
- CMD is now ["audio-refinery-service"]. CLI mode still works
  as an override: `docker run --gpus all <image> audio-refinery
  pipeline --help`.

.dockerignore picks up the new gitignored working dirs (_reqs/,
_plans/, _docs/) so the build context stays tight.

Makefile gains two targets:
- build-image reads the version from pyproject.toml and tags
  both lunarcommand/audio-refinery:<version> and :latest. The
  version pin keeps the tag in lock-step with releases; bumping
  pyproject.toml is the only place to change the image tag.
- run-service-local does a `docker run --gpus all` against
  bind-mounted /inbox + /outbox + /summaries dirs for local
  file:// dev. Fails fast if REFINERY_API_KEYS or HF_TOKEN
  isn't set in the operator's environment.

Tests unchanged: 353/353. Image build is a Phase 9 / Phase 12
concern (GPU host + multi-GB image); this commit just lands the
source artifacts that the build will consume.

* Add service-mode end-to-end and integration tests

Phase 9: two new test files cover the full service stack at
different levels.

tests/service/test_e2e.py (4 tests, no GPU, runs in CI):
- The pipeline is mocked but every other layer runs real. POST
  -> auth -> URI validation -> queue -> Worker thread -> URI I/O
  all execute against the real FastAPI app and Worker class.
- Covers happy path (transcript + summary written), failure
  surface (bad input -> no transcript, summary captures failure
  with retryable flag), multi-job submission order, and HTTPS
  end-to-end via httpx.MockTransport (verifies GET on input URL
  plus PUT on both transcript and summary URLs with JSON bodies
  that round-trip through the schema validators).

tests/service/test_integration.py (1 test, GPU-required):
- test_real_pipeline_file_uri_end_to_end runs the real pipeline
  against a known test WAV. Lifespan warm_up loads real pyannote
  + WhisperX + (optional) sentiment, POSTs a job via file://,
  waits up to 5 minutes for the batch, then asserts the
  transcript JSON is a valid CombinedTranscript with non-empty
  diarization + transcription + model_versions, and the summary
  JSON captures the success.
- Skips when the test WAV is missing or HF_TOKEN is unset.
- @pytest.mark.integration excludes it from default runs;
  triggered via make test-integration.

The e2e tests complement test_app.py (HTTP boundary in isolation)
and test_jobs.py (Worker in isolation) by exercising both
together. The integration test complements both by exercising
the real pipeline at the same time.

Full unit suite: 357/357. 6 integration tests deselected by
default (5 existing CLI + 1 new service).

* Make integration-test audio location configurable

The hardcoded /mnt/fast_scratch/test_fixtures/test_audio.wav path
was workstation-specific (a local RAM disk that may not be present
on other GPU hosts). Replace it with a configurable fixture so any
host with WAV files can run the integration suite.

tests/conftest.py adds two shared fixtures:
- integration_audio_files: returns every WAV in the configured
  directory, or skips cleanly with a clear message. Resolution
  order: REFINERY_TEST_AUDIO_DIR env var, then
  tests/_audio_fixtures/ in the repo (gitignored).
- integration_audio: single-file convenience wrapper that returns
  the first WAV. Existing tests that only need one file use this
  unchanged.

tests/test_integration.py drops its local fixture and the
hardcoded TEST_AUDIO_PATH constant. Existing 5 tests now use the
shared conftest fixture and skip with the clearer message when no
audio is configured.

tests/service/test_integration.py drops its local fixture and
adds a second test that exercises the batch-summary path against
the real pipeline:
- test_real_pipeline_multi_job_batch POSTs every WAV in the
  fixture dir as a single batch, waits for completion (timeout
  scales with input count), and asserts every transcript landed,
  the summary captures all jobs in submission order, and the
  totals are correct.
- Skips when fewer than 2 WAVs are available, since the single-
  job case is already covered by the existing
  test_real_pipeline_file_uri_end_to_end.
- Common spin-up logic extracted into _run_batch() so both tests
  share the warmup + POST + wait scaffolding.

.gitignore and .dockerignore add tests/_audio_fixtures/ so the
developer-local fallback dir doesn't leak into git or Docker
context.

Makefile help line updated to mention REFINERY_TEST_AUDIO_DIR
and HF_TOKEN; an echo before the pytest invocation reminds
operators of the requirements.

Usage:

    export REFINERY_TEST_AUDIO_DIR=/path/to/wavs
    export HF_TOKEN=hf_xxx
    make test-integration

Full unit suite: 357/357. 7 integration tests now skip cleanly
when fixtures are missing (was 6 — added the multi-job batch).

* Make service-mode Demucs scratch location configurable

Service mode was silently dropping the RAM-disk benefit the CLI
had. Each job's tempfile.TemporaryDirectory() landed in /tmp,
which in Docker is overlayfs (disk-backed) by default. Operators
on RAM-rich hosts had no clean knob to point scratch at tmpfs;
operators on RAM-tight VMs had no knob to point it at an SSD.

ServiceConfig gains scratch_dir: Path | None = None, sourced from
new REFINERY_SCRATCH_DIR env var. When set, process_job passes it
as `dir=` to tempfile.TemporaryDirectory(), so every job's input
download + Demucs stems + per-stage JSONs land under the operator-
chosen path. When unset (CLI test runs, dev setups), tempfile
falls back to TMPDIR / /tmp.

src/service/app.py:
- _resolve_scratch_location(config) returns (path, fstype).
- _detect_fstype(path) reads /proc/mounts and returns the
  filesystem type of the deepest mount containing path. None on
  non-Linux or unreadable /proc. Pure stdlib, no extra deps.
- service.ready log line includes scratch_dir + scratch_fstype +
  scratch_is_tmpfs so operators can verify from container logs
  what they got.
- When the detected fstype is not tmpfs, emit a scratch.not_tmpfs
  warning with a hint about mounting tmpfs and setting
  REFINERY_SCRATCH_DIR. Operators on RAM-tight VMs can ignore
  the warning intentionally.

Dockerfile:
- Pre-create /scratch owned by the refinery user.
- ENV REFINERY_SCRATCH_DIR=/scratch makes it the default location.
- VOLUME ["/scratch"] declares the path so the contract is
  visible in docker inspect and any operator scanning the image.
- Operators bind tmpfs at /scratch via:
    docker run --mount type=tmpfs,destination=/scratch ...
  or accept the disk-backed default (overlayfs) on tight hosts.

Tests:
- test_process_job_creates_per_job_temp_under_scratch_dir_when_configured
  asserts the worker honors scratch_dir.
- 4 tests for _resolve_scratch_location and _detect_fstype cover
  the config-set / config-None branches plus /proc/mounts parsing
  (synthetic fake mounts) and the non-Linux fallback to None.

Full unit suite: 362/362.

* Drop workstation-specific Demucs scratch default in CLI

The hardcoded /mnt/fast_scratch path in src/separator.py and the
interactive prompt UX in src/cli.py were specific to one
workstation's setup — same anti-pattern as the integration-test
audio path we just cleaned up.

Resolution rules now match service mode:
  1. REFINERY_SCRATCH_DIR env var (shared with the service).
  2. tempfile.gettempdir() / "audio-refinery-demucs" — host-agnostic
     default that works on laptops, containers, shared hosts.

src/separator.py:
- DEFAULT_OUTPUT_DIR is computed by _default_output_dir() at import
  time, honoring REFINERY_SCRATCH_DIR.
- No more hardcoded /mnt/fast_scratch.

src/cli.py:
- _resolve_demucs_scratch drops the /mnt/fast_scratch.is_mount()
  check and the interactive "RAM Disk Not Available" prompt. It
  now consults REFINERY_SCRATCH_DIR / DEFAULT_OUTPUT_DIR and
  detects whether the resolved path lives on tmpfs.
- _mkdir_demucs drops the interactive "RAM Disk Not Writable"
  PermissionError fallback prompt. A non-writable scratch dir is
  a real operator misconfiguration now (since the default works
  everywhere) and surfaces as a normal PermissionError.
- demucs_on_ramdisk detection moves from .is_mount() (which only
  works for explicit mount points) to detect_fstype() == "tmpfs"
  (works for any path on a tmpfs mount).
- The runtime banner shows "(RAM-backed)" or
  "(disk-backed — set REFINERY_SCRATCH_DIR to a tmpfs mount for
  faster batches)" instead of the workstation-specific
  "RAM disk not mounted" wording.

src/fs_utils.py is a new tiny stdlib-only module hosting
detect_fstype(path). Both src/service/app.py and src/cli.py
import it. Previously only service/app.py had a private copy;
extracting it removes the duplication and gives the CLI access
to the same logic.

Tests:
- tests/test_fs_utils.py is new, with the 2 tests that previously
  lived in tests/service/test_app.py (moved with the function).
- No CLI tests change because the old workstation prompt path
  wasn't exercised by tests (it was interactive-prompt code).

Full unit suite: 362/362.

Note: tests/service/test_e2e.py::test_e2e_https_uri_routes_*
flakes occasionally during the full-suite run but passes
consistently in isolation. Likely a TestClient lifespan teardown
timing issue interacting with httpx MockTransport state in
neighboring tests. Tracking as a known flake — not blocking.

* Fix flake in test_e2e_https_uri_routes_*

The test was patching run_pipeline AFTER the POST. The worker
thread is real and starts immediately when the TestClient lifespan
opens, so under load it could pull the queued job and call the
unpatched run_pipeline before the post-POST patch took effect —
the real run_pipeline then crashed against the MagicMock handles
and the assertions failed.

Use the dynamic-side-effect pattern that the multi-job test
already used: derive content_id from the source_dir glob at
call time, apply the patch in the OUTER `with` block before the
POST. The race window is closed: by the time the worker pulls
the job, every patch it needs is already in place.

Confirmed clean across 5 consecutive full-suite runs.

Note for posterity: this was a test-only race. Production POST
ordering is jobs.add → batches.add → queue.put_nowait, so the
worker can never pull a job whose registry records aren't
already populated.

* Gate POST /transcribe on service readiness

The endpoint accepted work during warmup, creating two problems:

1. Zombie jobs: if warmup ultimately failed (CUDA OOM, missing
   weights, bad HF token), every job queued before that point sat
   in the queue of a container the orchestrator was about to
   restart. Those jobs got dropped silently.
2. Split-brain readiness: /health reported "loading" / "failed"
   while /transcribe still answered 202. Operators and consumers
   reasonably expected the two surfaces to track.

Add a readiness gate at the top of POST /transcribe:
  - state="ready"   -> 202 (existing happy path)
  - state="loading" -> 503 with detail {state, stage}
  - state="failed"  -> 503 with detail {state, stage}
Response carries a Retry-After: 5 header so the caller's SQS
retry path can back off intentionally.

GET /jobs/{id} is deliberately NOT gated. The in-memory registry
is independent of pipeline readiness, and the endpoint stays
useful for operators debugging a stuck-loading container.

Lifespan fix: when create_app is called with handles pre-supplied
(test path, or any caller that warms outside the lifespan),
warm_up is skipped — but warm_up is also what normally calls
readiness.mark_ready(). Without an explicit mark in the
handles-supplied branch, readiness stayed "loading" and the new
gate rejected every request. Mark ready alongside starting the
background workers in that branch.

Tests:
- test_transcribe_returns_503_during_warmup
- test_transcribe_returns_503_when_warmup_failed
- _client() helper defaults to a ready ServiceReadiness so the
  happy-path tests don't have to opt in. Tests that exercise the
  gate pass an explicit loading/failed readiness.
- Two e2e tests had the same race the HTTPS test had (patching
  run_pipeline after the POST). Switched both to the dynamic-
  side-effect pattern that lets the patch sit in the outer `with`
  block. 10 consecutive full-suite runs clean.

Full unit suite: 364/364.

* Refresh CLI help text after scratch-location cleanup

The pipeline command's docstring still described the old
/mnt/fast_scratch + interactive-prompt UX from before the scratch
cleanup landed. Updated to describe the new resolution order:
REFINERY_SCRATCH_DIR/demucs (operator-configured tmpfs mount
preferred) -> tempfile.gettempdir()/audio-refinery-demucs.

Also:
- --demucs-dir help: drop "RAM disk check" phrasing; mention
  REFINERY_SCRATCH_DIR as the normal operator knob.
- pipeline-parallel scratch-suffix label: "(RAM-backed)" /
  "(disk-backed)" matches the wording the pipeline command's
  banner uses.
- Pipeline docstring: replaced "RAM disk" with "scratch
  directory" where the wording was assumptive.

No behavior change; help text only. Tests stay at 364/364.

* Drop required audio_ prefix on CLI source filenames

The pipeline's file discovery was hardcoded to match audio_<id>.wav,
a naming convention inherited from the upstream audio-extractor
tool. New OSS adopters with their own audio files were getting
"No audio_*.wav files found" against a directory full of legitimate
.wav files. The convention leaked into help text, error messages,
and the path helpers that mirror Demucs's output subdir.

CLI mode now accepts any .wav file. content_id is the filename
stem with an optional audio_ prefix stripped, so previous runs
that wrote diarization_<id>.json keep landing in the same place
after the rename.

src/pipeline.py:
- discover_files: glob("*.wav"); content_id = stem.removeprefix("audio_").
- _vocals_path / _no_vocals_path: take input_stem (the actual
  filename stem Demucs derives its subdir from) instead of
  content_id. Demucs subdir matches the input filename whether
  or not it has the audio_ prefix.
- All call sites updated to pass wav_path.stem.
- Stage runners (run_diarization_stage, run_transcription_stage)
  now capture wav_path from the iterated tuple instead of
  discarding it with _.

src/service/jobs.py:
- Service mode creates the per-job audio file as <content_id>.wav
  (no audio_ prefix). Equivalent to what came before, just one
  fewer indirection in the filename.
- _content_id_from_job_id docstring no longer claims the prefix
  is required.

src/cli.py:
- "No audio_*.wav files found" -> "No .wav files found"
- "Create the directory and place audio_<content_id>.wav files"
  -> "place .wav files"

tests/test_pipeline.py:
- source_dir fixture creates <cid>.wav instead of audio_<cid>.wav.
- test_discover_files_ignores_non_matching split into:
  - test_discover_files_ignores_non_wav_extensions (.mp3 still
    skipped)
  - test_discover_files_accepts_arbitrary_stem (proves any stem
    works, including dashes and mixed case)
  - test_discover_files_strips_audio_prefix_for_backward_compat
    (proves the prefix-strip backward-compat path)
- _sep_side_effect helpers no longer slice off audio_ since the
  stem doesn't have it.

tests/service/test_e2e.py + test_jobs.py:
- Dynamic-side-effect helpers glob "*.wav" instead of "audio_*.wav".
- Synthetic audio paths in mock pipeline outputs use <content_id>.wav.

Full unit suite: 366/366 across 5 consecutive runs. Two new tests
explicitly cover the relaxed behavior; existing tests cover the
backward-compat prefix-strip path.

* Treat silent input as a sentiment no-op, not a failure

analyze_sentiment() raised SentimentError("No usable text found in
transcription segments") when the transcription had zero usable
text. That collapsed two distinct outcomes into the same failure
path:

1. Silent input — the file genuinely had no speech, so Whisper
   returned no segments (or empty segments). Not a real error;
   sentiment ran successfully, it just had nothing to score.
2. Pipeline crashed on every segment — the segments had text but
   every classifier call raised. Actual failure.

Distinguish them: return an empty SentimentResult for case 1,
keep raising for case 2. The pipeline's Failed table no longer
flags silent inputs; the Slack summary stops saying "complete
with 1 failure(s)" when sentiment found nothing to score.

Tests:
- test_no_segments_returns_empty_result (was _raises_sentiment_error)
- test_all_empty_text_returns_empty_result (same flip)
- test_all_segments_fail_classification_raises (new — proves the
  real-failure path still raises with the matching message)

The merge_sentiment_into_transcription helper handles empty
results correctly already (it just writes no fields per segment).
Full unit suite: 367/367.

* Resolve symlinks when predicting Demucs output paths

src/separator.py:separate() calls Path(input_file).resolve() before
invoking Demucs, so Demucs writes its per-track subdir using the
resolved file's stem — not the symlink's name. The pipeline's
_vocals_path / _no_vocals_path callers used wav_path.stem (the
symlink's name when the input was symlinked), producing a path
mismatch the diarization stage surfaced as:

  Diarization failed for <id>: Input file not found:
  /tmp/.../stems/htdemucs/<symlink-stem>/vocals.wav

This bit service mode in particular: process_job symlinks the
fetched input into the per-job source_dir as <content_id>.wav,
where content_id is derived from job_id. The real file's stem
(e.g., the original WAV name) is different, so Demucs's actual
output landed at htdemucs/<original-stem>/ while the pipeline
looked at htdemucs/<content_id>/.

Fix: change all 7 _vocals_path / _no_vocals_path callers in
pipeline.py to pass wav_path.resolve().stem. For regular files
(typical CLI mode) this is a no-op — resolve() returns the file
itself. For symlinks (service mode, or any CLI user whose
extracted/ contains symlinks) it follows the link the same way
separate() does, so the predicted path matches Demucs's actual
output.

The helper signatures stay string-based; the .resolve() lives at
the call sites where wav_path is in scope.

New test test_run_pipeline_resolves_symlinks_for_demucs_output_paths
sets up a symlink with a different stem from the target, mocks
separate() to write to the resolved name (mirroring real Demucs
behavior), and asserts the diarization stage doesn't fail with a
missing-vocals error. Pinned so we don't regress this.

Full unit suite: 368/368.

* Force UTF-8 decoding of Demucs subprocess output

subprocess.run(cmd, capture_output=True, text=True) decodes stdout
and stderr using locale.getpreferredencoding(False). On hosts
whose locale resolves to ASCII — common in minimal container
images and some CI / service runners — Demucs's tqdm progress
bars (which use Unicode glyphs ━, █, U+2501 etc.) crash the
decoding with:

  'ascii' codec can't decode byte 0xe2 in position 127:
  ordinal not in range(128)

The error bubbles out of subprocess.run before separate() can
even check the returncode. Demucs's actual separation work
already finished and the stems are on disk; the failure is
purely in capturing stderr.

Specify encoding="utf-8", errors="replace" explicitly. errors=
"replace" maps any genuinely undecodable byte sequence to
U+FFFD rather than raising — losing a few progress-bar chars
to replacement is harmless, while crashing the entire job is
not. text=True and capture_output=True stay as-is.

This bit service mode in particular: the worker hit it on the
second job in a batch (the first apparently got lucky with
output timing). CLI mode worked on the same files for the same
reason — Demucs's stderr varies between invocations.

Regression test pins the kwargs separate() passes to
subprocess.run so we don't drift back.

Full unit suite: 369/369.

* Pin Dockerfile uv installs to python3.11

ubuntu:22.04 ships python3.10 as the default python3. The base
image installs python3.11 alongside via apt, but uv's --system
flag resolves to whatever python3 points at (3.10), then fails
dependency resolution:

  Because the current Python version (3.10.12) does not satisfy
  Python>=3.11,<3.12 and audio-refinery==0.1.1 depends on
  Python>=3.11,<3.12, audio-refinery==0.1.1 cannot be used.

Set UV_PYTHON=python3.11 once, before the first `uv pip install`,
so every install in this Dockerfile targets the 3.11 site-packages
that pyproject.toml requires. uv accepts bare interpreter names
and resolves via PATH; no need to thread --python through each
command.

`pip install --user uv` still runs against python3 (3.10),
which is fine because uv is a Rust binary that doesn't depend on
the installer's Python.

Image build was failing at:
  [ 7/11] RUN uv pip install --system -e .
This fix unblocks `make build-image` so Test 6 can proceed.

* Fix Docker build perms and include README

The previous Dockerfile switched to the refinery user before any pip
installs ran, but `uv pip install --system` writes to root-owned
`/usr/lib/python3.11/site-packages`, and the editable install needs
to write `audio_refinery.egg-info` into `/app`. Move all installs to
root and switch to refinery only for the runtime.

Also un-ignore README.md so setuptools stops warning about a missing
file referenced from pyproject.toml's `readme` field.

* Add `audio-refinery serve` CLI subcommand

Expose the HTTP service under the same binary users already know, so
local-dev workflows need only one entry point name. It lazy-imports
src.service.app.run, keeping uvicorn/FastAPI off the import path for
every other CLI invocation. The production container CMD still calls
audio-refinery-service directly.

* Rename docs/ files to lowercase

Adopt lowercase filenames inside docs/ ahead of the dual-mode docs
restructure, reserving uppercase for the root meta files (README,
CHANGELOG, etc.) per UNIX/GitHub convention. Updates every live
cross-reference; the [0.1.1] CHANGELOG entry keeps its original
DEPLOYMENT.md spelling as a historical record.

* Add docs/cli.md and slim README to a two-path landing

Extract the full per-command CLI manual into docs/cli.md and reduce the
README to a high-level overview, a "choose your path" landing for the
CLI and service modes, shared install/token setup, and the docs table.

Correct stale references while extracting: the default Demucs scratch
dir is /tmp/audio-refinery-demucs (REFINERY_SCRATCH_DIR override), batch
input accepts any *.wav with the audio_ prefix optional, and the
scaffolded --emotion/--events pipeline flags are documented.

* Add docs/service.md operational guide

Document the HTTP service mode end to end: quickstart, the three
endpoints with status codes, combined-transcript and batch-summary
schemas (v1.0.0), the full REFINERY_* / HF_TOKEN / SLACK env table,
URI schemes, readiness-probe and scaling ops, the file:// dev loop,
and troubleshooting. Field names, defaults, and status codes are
drawn from the service code so the guide stays authoritative.

* Add docs index hub and service notes to existing docs

Add docs/index.md as the navigation hub. Document service mode in the
guides that contributors read: development.md gains "Running the
Service Locally" and a service-tests note, CLAUDE.md lists the serve
subcommand and the FastAPI/uvicorn/httpx deps, and CONTRIBUTING.md
flags that run_pipeline's optional model_handles parameter must stay
intact for service mode.

* Rework deployment.md around CLI and service modes

Split the guide into CLI deployment (workstation batch) and service
deployment (containerized HTTP API). Replace the stale embedded
Dockerfile with a pointer to the repo Dockerfile and make build-image,
and replace the DIY PostgreSQL worker-queue pattern — now superseded by
service mode — with service deployment guidance cross-linked to
service.md. Correct the scratch-directory default and keep the
prerequisites, monitoring, and VRAM sections.

* Document execution modes and fix stale doc references

Add an "Execution Modes: CLI and Service" section to architecture.md
so it covers the core/CLI/service split the README and index promise,
correct the stale audio_<id>.wav input-naming note in use-cases.md to
match the any-*.wav behavior, and point the service dev loop at the
make run-service-local target.

* Prepare release v0.2.0

Bump the version to 0.2.0 and promote the CHANGELOG [Unreleased]
section, documenting service mode, the audio-refinery serve
subcommand, the host-agnostic scratch directory, the relaxed batch
input naming, and the docs restructure.

Add a docker-publish job to the release workflow so a v*.*.* tag also
builds and pushes the image to Docker Hub. The job needs
DOCKERHUB_USERNAME and DOCKERHUB_TOKEN repo secrets; without them it
fails at login while the GitHub release still succeeds.

* Address PR review feedback for v0.2.0

Substantive fixes:
- Make all service-mode timestamps UTC-aware via a shared _utcnow()
  helper, so the combined transcript no longer mixes naive and
  aware datetimes and the retention cutoff compares correctly.
- Redact the full bearer token in the access-log filter with a
  regex instead of slicing a fixed 16-character window.
- Derive the FastAPI app version from package metadata instead of a
  hardcoded 0.2.0-dev, so it tracks the release.
- Align the standalone separate scratch default to
  $REFINERY_SCRATCH_DIR/demucs, matching the pipeline and docs.

Cleanup:
- Remove the unused JobStatus documentation alias in jobs.py.
- Drop a dead Registries assignment and move test delete() calls out
  of assert expressions so they run under python -O.